rnmamod: An R Package for Conducting Bayesian Network Meta-analysis with Missing Participants

The development of several R packages for conducting network meta-analysis has enhanced the popularity of this evidence synthesis tool. The available R packages facilitate the implementation of most models to conduct and evaluate network meta-analysis and provide the necessary results, conforming to the PRISMA-NMA statement. The rnmamod package is a novel contribution to conducting aggregate network meta-analysis using Bayesian methods, as it allows addressing missing participants properly in all models, even if a handful of the included studies report this information. Importantly, rnmamod is the first R package to offer a rich, user-friendly visualisation toolkit that turns a “parameter-dense” output of network meta-analysis into several comprehensive graphs. Furthermore, the package functions on various models allow processing their output to create visualisations tailored to the user preferences. Therefore, rnmamod aids the thorough appraisal and interpretation of the results, the cross-comparison of different models and the manuscript preparation for journal submission.

Loukia M. Spineli https://www.github.com/LoukiaSpin (Midwifery Research and Education Unit) , Chrysostomos Kalyvas https://www.github.com/ckalyvas (Biostatistics and Research Decision Sciences) , Katerina Papadimitropoulou https://www.github.com/Katerina-Pap (Health Economics and Market Access)
2022-11-22

1 Introduction

Evidence-based medicine is the backbone of informed decisions for the benefit of the patients, stemming from a meticulous and judicious use of the available evidence, while taking into account also the clinical experience and patient values (Sackett et al. 1996). However, the medical community is faced daily with several intervention options and dosages, challenging the optimal practice of evidence-based medicine (Lee 2022). Systematic reviews with pairwise meta-analysis summarise the evidence of pairs of interventions, providing fragmented evidence that does not serve the clinical needs. Moreover, evidence in the comparability of different interventions at the trial level is also fragmented, as it is not feasible to compare all intervention options for a condition in one trial. These limitations led to the development and later establishment of network meta-analysis (NMA), also known as multiple treatment comparison, a new generation evidence synthesis tool (Salanti 2012). Network meta-analysis is an extension of pairwise meta-analysis for collecting all relevant pieces of evidence for a specific condition, patient population, and intervention options to provide coherent evidence for all possible intervention comparisons, and allow ordering the investigated interventions from the best to worst option for a specific outcome (Caldwell 2014). Indirect evidence (obtained from different sets of trials sharing a common comparator) plays a central role in the development and prominence of NMA.

Since the introduction of indirect evidence and early development of the relevant methodology (Higgins and Whitehead 1996; Bucher et al. 1997), the NMA framework has undergone substantial progress conceptually and methodologically. The fast-pace publications of relevant methodological articles and systematic reviews with NMA attest to the increasing popularity of NMA in the wide medical and evidence synthesis community (Efthimiou et al. 2016; Petropoulou et al. 2017). Needless to say that the availability of statistical analysis software is the driving force to the advances and wide dissemination of NMA. A review of the methodology and software for NMA (Efthimiou et al. 2016) listed several statistical software tools used to promote NMA, with the R software (R Core Team 2022) being the most popular to develop and compare methods for NMA, followed by Stata (StataCorp 2021) and SAS software (SAS Institute 2020).

In the last decade, there has been a raise in the R packages for NMA with various functionalities (Dewey and Viechtbauer 2022). These packages can be categorised by, among others:

Most packages fall into many categories. For instance, gemtc (van Valkenhoef and Kuiper 2021), probably the most popular R package for Bayesian NMA, allows both for one-stage and two-stage approaches using contrast-based modeling, has a wide scope, and deals with aggregate outcome data of many types. netmeta (Rücker et al. 2022) is currently the only R package developed exclusively for NMA in the frequentist framework based on the graph theory (Ruecker 2012), allows only for a two-stage approach (contrast-based modeling), has also a wide scope, and accommodates binary, rates, and continuous aggregate outcome data. On the other side, R packages, such as nmathresh (Phillippo et al. 2018), nmaplateplot (Wang et al. 2021), and nmarank (Nikolakopoulou et al. 2021) do not perform NMA, but use the NMA results (obtained using other R packages or statistical software tools) as an input to provide, for instance, decision-invariant bias-adjustment thresholds and intervals (nmathresh (Phillippo et al. 2018)), various league tables in heatplot style with all intervention comparisons (nmaplateplot (Wang et al. 2021)), or an intervention hierarchy approach tailored to the research question (nmarank (Nikolakopoulou et al. 2021)).

Due to the complexity and the wide scope of NMA, the researchers are faced with a large volume of results, necessary to understand the evidence base, assess the underlying assumptions, evaluate the quality of the estimated parameters (model diagnostics), and properly answer the research question, for instance, concerning the comparative effectiveness of the competing interventions and their hierarchy. To address the challenges associated with the best reporting of NMA results, the PRISMA-NMA statement (Hutton et al. 2015) was developed expanding on the PRISMA statement for pairwise meta-analysis (Page et al. 2021) to provide an extensive checklist with the essential items pertaining to the NMA results, ensuring completeness in the reporting of systematic reviews with multiple interventions. The R packages PRISMAstatement (Wasey 2019) and metagear (Lajeunesse 2021) facilitate the creation of the PRISMA flow chart and the process of article screening and data extraction, conforming to the PRISMA statement (Page et al. 2021), and are also relevant for systematic reviews with multiple interventions. The additional items in the PRISMA-NMA statement that apply to the NMA framework, such as presentation and summary of network geometry, inconsistency assessment, league tables and presentation of intervention hierarchy, are addressed in most R packages either in a targeted manner (e.g., nmaplateplot (Wang et al. 2021), and nmarank (Nikolakopoulou et al. 2021)) or collectively (bnma (Seo and Schmid 2022), netmeta (Rücker et al. 2022), gemtc (van Valkenhoef and Kuiper 2021), and pcnetmeta (Lin et al. 2017)).

Most methodological studies on and systematic reviews with NMA have implemented Bayesian methods (Efthimiou et al. 2016; Petropoulou et al. 2017). The advantages of the Bayesian framework (e.g., flexible modeling, allowance of uncertainty in all model parameters, incorporation of external relevant information and facilitation of probabilistic statements) (Sutton and Abrams 2001), in conjunction with the dominance of the BUGS software (Lunn et al. 2009) during the springtime of the NMA framework, may be attributed to the popularity of Bayesian NMA. The rise of R packages on Bayesian NMA also demonstrates the acclaim of Bayesian methods from the evidence synthesis community. The rest of the section pertains to R packages on Bayesian NMA published in the CRAN Task View ‘Meta-Analysis’ (Dewey and Viechtbauer 2022) that feature a wide methodological and reporting scope: bnma (Seo and Schmid 2022), gemtc (van Valkenhoef and Kuiper 2021), pcnetmeta (Lin et al. 2017), and rnmamod (Spineli 2022) (a recent novel contribution).

The R packages bnma (Seo and Schmid 2022), gemtc (van Valkenhoef and Kuiper 2021), and pcnetmeta (Lin et al. 2017) conduct hierarchical NMA using Markov chain Monte Carlo methods through the JAGS program (Plummer 2003). However, these packages differ in their methodological and reporting breadth to some extent: bnma (Seo and Schmid 2022) and gemtc (van Valkenhoef and Kuiper 2021) have a greater common basis on methods and outputs than pcnetmeta (Lin et al. 2017). This may be attributed to using the contrast-based modeling approach, which is the established approach to meta-analysis (trial-specific relative treatment effects, such as log odds ratio, are pooled across the trials), whilst pcnetmeta (Lin et al. 2017) considers the arm-based modeling approach, which deviates from the standard meta-analysis practice (Dias and Ades 2016) and is less widespread. Currently, the package pcnetmeta (Lin et al. 2017) does not contain any function to conduct inconsistency evaluation and meta-regression, is limited only to rankograms in terms of hierarchy measures (Salanti et al. 2022), and considers only the trace plots as a visual diagnostic tool. On the contrary, bnma (Seo and Schmid 2022) and gemtc (van Valkenhoef and Kuiper 2021) offer at least one method for inconsistency evaluation, allow conducting meta-regression and consider a wider variety of hierarchy measures and diagnostic tools. However, all three R packages provide a small-sized toolkit with functions regarding the presentation of the relative treatment effects: a league table for one outcome that appears only in the console, and a forest-plot or table on the relative treatment effects of all comparisons with the selected intervention.

Furthermore, other common features of the R packages bnma (Seo and Schmid 2022), gemtc (van Valkenhoef and Kuiper 2021), and pcnetmeta (Lin et al. 2017) include: (i) the undue reliance on the function print() (the results appear in the console) than visualisation, and (ii) presenting the results mostly in isolation, restricting the ability to gain further insights into the performance of the NMA models and contextualise the results in the light of the strengths and limitations in the analysis. The package

Table 1: Features of R packages for network meta-analyses (CRAN Task View)
Package
Analysis
Modeling approach
Scope breadth
Outcome structure
Bayesian Frequentist Contrast Arm Wide Narrow AD & IPD AD
bnma X X X X
gemtc X X X X
metapack X X X X
multinma1 X X X X
netmeta X X X X
NMADiagT X X X X
nmaINLA2 X X X X
NMAoutlier X X X X
nmaplateplot X X X X X
nmarank3 X X X X X
nmathresh4 X X X X X
pcnetmeta X X X X
Note:
AD, aggregate data; IPD, individual patient data.
1 Estimation using the probabilistic programming language Stan.
2 Use of integrated nested Laplace approximation.
3 Mainly frequentist-driven but can be easily applied to Bayesian results.
4 Mainly Bayesian-driven but can be naturally applied to the frequentist framework.

rnmamod provides ‘dynamic’ graphs by combining plots.

2 Background

Some packages on interactive graphics include plotly (Sievert 2020) that interfaces with Javascript for web-based interactive graphics, crosstalk (Cheng and Sievert 2021) that specializes cross-linking elements across individual graphics. The recent R Journal paper tsibbletalk (Wang and Cook 2021) provides a good example of including interactive graphics into an article for the journal. It has both a set of linked plots, and also an animated gif example, illustrating linking between time series plots and feature summaries.

3 Customizing tooltip design with ToOoOlTiPs

ToOoOlTiPs is a packages for customizing tooltips in interactive graphics, it features these possibilities.

4 A gallery of tooltips examples

The palmerpenguins data (Horst et al. 2020) features three penguin species which has a lovely illustration by Alison Horst in Figure 1.

A picture of three different penguins with their species: Chinstrap, Gentoo, and Adelie.

Figure 1: Artwork by @allison_horst

Table 2 prints at the first few rows of the penguins data:

Table 2: A basic table
species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g sex year
Adelie Torgersen 39.1 18.7 181 3750 male 2007
Adelie Torgersen 39.5 17.4 186 3800 female 2007
Adelie Torgersen 40.3 18.0 195 3250 female 2007
Adelie Torgersen NA NA NA NA NA 2007
Adelie Torgersen 36.7 19.3 193 3450 female 2007
Adelie Torgersen 39.3 20.6 190 3650 male 2007

Figure 2 shows an interactive plot of the penguins data, made using the plotly package.

p <- penguins %>% 
  ggplot(aes(x = bill_depth_mm, y = bill_length_mm, 
             color = species)) + 
  geom_point()
ggplotly(p)

Figure 2: A basic interactive plot made with the plotly package on palmer penguin data. Three species of penguins are plotted with bill depth on the x-axis and bill length on the y-axis. When hovering on a point, a tooltip will show the exact value of the bill depth and length for that point, along with the species name.

5 Summary

We have displayed various tooltips that are available in the package ToOoOlTiPs.

CRAN packages used

netmeta, NMAoutlier, bnma, gemtc, metapack, multinma, NMADiagT, nmaINLA, pcnetmeta, nmaplateplot, nmarank, nmathresh, PRISMAstatement, metagear, rnmamod, plotly, crosstalk, tsibbletalk, palmerpenguins, ggplot2

CRAN Task Views implied by cited packages

MetaAnalysis, MissingData, Phylogenetics, Spatial, TeachingStatistics, TimeSeries, WebTechnologies

H. C. Bucher, G. H. Guyatt, L. E. Griffith and S. D. Walter. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol, 50(6): 683–691, 1997. DOI 10.1016/s0895-4356(97)00049-8.
D. M. Caldwell. An overview of conducting systematic reviews with network meta-analysis. Syst Rev, 3: 109, 2014. DOI 10.1186/2046-4053-3-109.
J. Cheng and C. Sievert. crosstalk: Inter-widget interactivity for HTML widgets. 2021. URL https://CRAN.R-project.org/package=crosstalk. R package version 1.1.1.
M. Dewey and W. Viechtbauer. CRAN Task View: Meta-analysis. 2022. URL https://CRAN.R-project.org/view=MetaAnalysis. Version 2022-11-05.
S. Dias and A. E. Ades. Absolute or relative effects? Arm-based synthesis of trial data. Res Synth Methods, 7(1): 23–28, 2016. DOI 10.1002/jrsm.1184.
O. Efthimiou, T. P. A. Debray, G. van Valkenhoef, S. Trelle, K. Panayidou, K. G. M. Moons, J. B. Reitsma, A. Shang, G. Salanti and GetReal Methods Review Group. GetReal in network meta-analysis: A review of the methodology. Res Synth Methods, 7(3): 236–263, 2016. DOI 10.1002/jrsm.1195.
B. K. Guenhan. nmaINLA: Network meta-analysis using integrated nested laplace approximations. 2021. URL https://CRAN.R-project.org/package=nmaINLA. R package version 1.1.0.
J. P. Higgins and A. Whitehead. Borrowing strength from external trials in a meta-analysis. Stat Med, 15(24): 2733–2749, 1996.
A. M. Horst, A. P. Hill and K. B. Gorman. palmerpenguins: Palmer archipelago (antarctica) penguin data. 2020. URL https://allisonhorst.github.io/palmerpenguins/. R package version 0.1.0.
B. Hutton, G. Salanti, D. M. Caldwell, A. Chaimani, C. H. Schmid, C. Cameron, J. P. A. Ioannidis, S. Straus, K. Thorlund, J. P. Jansen, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann Intern Med, 162(11): 777–784, 2015. DOI 10.7326/M14-2385.
M. J. Lajeunesse. Metagear: Comprehensive research synthesis tools for systematic reviews and meta-analysis. 2021. URL https://CRAN.R-project.org/package=metagear. R package version 0.7.
A. Lee. The development of network meta-analysis. J R Soc Med, 115(8): 313–321, 2022. DOI 10.1177/01410768221113196.
D. Lim, M.-H. Chen, S. Kim, J. Ibrahim, A. Shah and J. Lin. Metapack: Bayesian meta-analysis and network meta-analysis. 2022. URL https://CRAN.R-project.org/package=metapack. R package version 0.2.0.
L. Lin, J. Zhang, J. S. Hodges and H. Chu. Performing arm-based network meta-analysis in R with the pcnetmeta package. Journal of Statistical Software, 80(5): 1–25, 2017. DOI 10.18637/jss.v080.i05.
B. Lu, Q. Lian, J. S. Hodges, Y. Chen and H. Chu. NMADiagT: Network meta-analysis of multiple diagnostic tests. 2020. URL https://CRAN.R-project.org/package=NMADiagT. R package version 0.1.2.
D. Lunn, D. Spiegelhalter, A. Thomas and N. Best. The BUGS project: Evolution, critique and future directions. Stat Med, 28(25): 3049–3067, 2009. DOI 10.1002/sim.3680.
A. Nikolakopoulou, G. Schwarzer and T. Papakonstantinou. Nmarank: Complex hierarchy questions in network meta-analysis. 2021. URL https://CRAN.R-project.org/package=nmarank. R package version 0.2-3.
M. J. Page, J. E. McKenzie, P. M. Bossuyt, I. Boutron, T. C. Hoffmann, C. D. Mulrow, L. Shamseer, J. M. Tetzlaff, E. A. Akl, S. E. Brennan, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372: n71, 2021. DOI 10.1136/bmj.n71.
T. Papakonstantinou, G. Salanti, D. Mavridis, G. Rücker, G. Schwarzer and A. Nikolakopoulou. Answering complex hierarchy questions in network meta-analysis. BMC Med Res Methodol, 22(1): 47, 2022. DOI 10.1186/s12874-021-01488-3.
M. Petropoulou, A. Nikolakopoulou, A.-A. Veroniki, P. Rios, A. Vafaei, W. Zarin, M. Giannatsi, S. Sullivan, A. C. Tricco, A. Chaimani, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol, 82: 20–28, 2017. DOI 10.1016/j.jclinepi.2016.11.002.
M. Petropoulou, G. Schwarzer, A. Panos and D. Mavridis. NMAoutlier: Detecting outliers in network meta-analysis. 2027. URL https://CRAN.R-project.org/package=NMAoutlier. R package version 0.1.18.
D. M. Phillippo. Multinma: Bayesian network meta-analysis of individual and aggregate data. 2022. URL https://CRAN.R-project.org/package=multinma. R package version 0.5.0.
D. M. Phillippo, S. Dias, A. E. Ades, V. Didelez and N. J. Welton. Sensitivity of treatment recommendations to bias in network meta-analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 181(3): 843–867, 2018. DOI 10.1111/rssa.12341.
M. Plummer. JAGS: A program for analysis of bayesian graphical models using gibbs sampling. Technische Universität Wien, Vienna, Austria, 2003. URL https://www.R-project.org/conferences/DSC-2003/Proceedings/Plummer.pdf.
R Core Team. R: A Language and Environment for Statistical Computing. Foundation for statistical computing, vienna, austria. 2022. URL https://www.R-project.org/.
G. Rücker, U. Krahn, J. König, O. Efthimiou, A. Davies, T. Papakonstantinou and G. Schwarzer. Netmeta: Network meta-analysis using frequentist methods. 2022. URL https://CRAN.R-project.org/package=netmeta. R package version 2.6-0.
G. Ruecker. Network meta-analysis, electrical networks and graph theory. Res Synth Methods, 3(4): 312–324, 2012. DOI 10.1002/jrsm.1058.
D. L. Sackett, W. M. Rosenberg, J. A. Gray, R. B. Haynes and W. S. Richardson. Evidence based medicine: What it is and what it isn’t. BMJ, 312(7023): 71–72, 1996. DOI 10.1136/bmj.312.7023.71.
G. Salanti. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: Many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods, 3(2): 80–97, 2012. DOI 10.1002/jrsm.1037.
G. Salanti, A. Nikolakopoulou, O. Efthimiou, D. Mavridis, M. Egger and I. R. White. Introducing the treatment hierarchy question in network meta-analysis. Am J Epidemiol, 191(5): 930–938, 2022. DOI 10.1093/aje/kwab278.
SAS Institute. The SAS System for Windows. Release 9.4. Cary, NC: SAS inst. 2020. URL https://www.sas.com.
M. Seo and C. Schmid. Bnma: Bayesian network meta-analysis using ’JAGS’. 2022. URL https://CRAN.R-project.org/package=bnma. R package version 1.5.0.
C. Sievert. Interactive Web-Based Data Visualization with r, plotly, and shiny. Chapman; Hall/CRC, 2020. URL https://plotly-r.com.
L. M. Spineli. Rnmamod: Bayesian network meta-analysis with missing participants. 2022. URL https://CRAN.R-project.org/package=rnmamod. R package version 0.3.0.
StataCorp. Stata Statistical Software: Release 17. College station, TX: StataCorp LLC. 2021. URL http://www.stata.com.
A. J. Sutton and K. R. Abrams. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res, 10(4): 277–303, 2001. DOI 10.1177/096228020101000404.
G. van Valkenhoef and J. Kuiper. Gemtc: Network meta-analysis using bayesian methods. 2021. URL https://CRAN.R-project.org/package=gemtc. R package version 1.0-1.
E. Wang and D. Cook. Conversations in time: Interactive visualisation to explore structured temporal data. The R Journal, 2021. URL https://journal.r-project.org/archive/2021/RJ-2021-050/index.html.
Z. Wang, L. Lin, S. Zhao and H. Chu. Nmaplateplot: The plate plot for network meta-analysis results. 2021. URL https://CRAN.R-project.org/package=nmaplateplot. R package version Haitao Chu.
J. O. Wasey. PRISMAstatement: Plot flow charts according to the "PRISMA" statement. 2019. URL https://CRAN.R-project.org/package=PRISMAstatement. R package version 1.1.1.

References

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".

Citation

For attribution, please cite this work as

Spineli, et al., "rnmamod: An R Package for Conducting Bayesian Network Meta-analysis with Missing Participants", The R Journal, 2022

BibTeX citation

@article{rnmamod-article,
  author = {Spineli, Loukia M. and Kalyvas, Chrysostomos and Papadimitropoulou, Katerina},
  title = {rnmamod: An R Package for Conducting Bayesian Network Meta-analysis with Missing Participants},
  journal = {The R Journal},
  year = {2022},
  note = {https://doi.org/10.32614/rnmamod-article},
  doi = {10.32614/rnmamod-article},
  issn = {2073-4859},
  pages = {1}
}